As a data combining apparatus for combining still image data, moving image data, or graphics data to form a screen, there is known, for example, a computer graphics system for synchronizing color data, depth data, texture data, or other data processed asynchronously (refer to Patent Document 1: Japanese Unexamined Patent Publication (Kokai) No. 9-6973) or a computer graphics system for combining a plurality of three-dimensional images in a two-dimensional graphics image processing pipeline and finally for performing rendering of the combined image on a display screen (refer to Patent Document 2: Japanese Unexamined Patent Publication (Kokai) No. 2001-357410).
FIG. 1 is a block diagram of an example of the configuration of a conventional data combining apparatus combining still image data, moving image data, or graphics data etc. to form a screen.
The data formation apparatus 10, as shown in FIG. 1, has a screen formation control unit 11, a graphics control unit 12, a graphics generation unit 13, an image control unit 14, a transformation and movement control unit 15, an image processing unit 16, and a combining unit 17.
The screen formation control unit 11 controls when and how data to be combined should be transformed (enlarged or reduced) and where the result should be moved (arranged) and combined on a screen.
The graphics control unit 12 controls the graphics generation unit 13 and generates graphics to be needed for a combined screen instructed from the screen formation control unit 11. The generated graphics S13 to be combined is input to the combining unit 17.
On the other hand, the image control unit 14 controls the transformation and movement processing unit 15 of the image and transforms and moves an input image data SIM to correspond to the combined screen instructed from the screen formation control unit 11. The transformed and moved image data S15 may be further processed by the image processing unit 16.
The image processing unit 16 performs, for example, filtering for an improvement of image quality and conversion of the image format. Note that, the image processing unit 16 sometimes is not provided. Also, similarly, a graphics processing unit not shown in the figures may be placed at the latter stage of the graphics generation unit 13.
The transformed and moved image data S15 is input to the combining unit 17 as image data S16 to be combined.
The combining unit 17 combines the input graphics S13 to be combined and the input image data S16 to be combined to output the combined data S17.
For example, a TV monitor or a PC monitor forms a menu provided to the user as a graphics, superposes the same on image data to be displayed, and displays the result. Such a system has also been called an “on-screen-display (OSD)”.
Further, the data combining apparatus shown in FIG. 1 is used in a service such as digital television broadcasting in which image data and graphics data are transmitted and a screen formation formed by combining the both is provided.
On the other hand, the combination system 10 shown in FIG. 1 does not strictly adjust the timing of the graphics S13 to be combined and the image data S16 to be combined.
Therefore, in the conventional data combining apparatus, there are disadvantages that it cannot accurately realize a screen formation such that transformation and movement (arrangement) of the data to be combined are included and specifically such that the amount of transformation and movement and arrangement position change continuously.
This point will be further considered with reference to FIG. 2A to FIG. 2C and FIG. 3A to FIG. 3C.
FIG. 2A to FIG. 2C are views showing an example of combination of the screen formation such that transformation and movement (arrangement) of the data to be combined are included and specifically the amount of transformation and movement and the arrangement position change continuously. FIG. 2A shows the graphics S13 to be combined, FIG. 2B shows the image data S16 to be combined, and FIG. 2C shows the combined data S17 respectively.
As shown in the figures, at the time T1, the transformation (reduction) and movement of the image data S16 to be combined is started.
In accordance with image data to be moved after the reduction, available graphics is generated at a region in which the image data does not exist.
In the example shown in FIG. 2A to FIG. 2C, the image data S16 and S17 are continuously reduced and moved along with the elapse. The available graphics region changes due to this.
As long as the image data S16 to be combined and the graphics S13 to be combined are combined at an accurate timing, the two regions are made correspondence accurately.
On the other hand, if the timing of the transformation and movement of the image data and the timing of the generation of the graphics deviate from each other, desirable combined data cannot be achieved.
FIG. 3A to FIG. 3C are views showing an example of combination in the case where the start of the transformation and movement of the image data is delayed by one frame's worth of time. FIG. 3A shows the graphics S13 to be combined, FIG. 3B shows the image data S16 to be combined, and FIG. 3C shows the combined data S17 respectively.
As shown in the figures, at the time T1, the change in the screen formation is started, and the graphics is generated at the accurate timing.
The example shown in FIG. 3A to FIG. 3C shows an example in which the start of the transformation (reduction) and movement of the image data to be combined is delayed at the time T2.
At the time T1, the image data S16 in which the change of the screen formation has not been started and the graphics S13 in which the same has been started are input to the combining unit 17, so that contradictory combined data is output.
Generally, since the graphics is superposed and combined at the front of the image data in the OSD, FIG. 3A to FIG. 3C show the case of superposing the graphics at the front of the image data. Similarly to this, in the case of superposing the image data at the front of the graphics, the contradictory combined data is output.
Further, the delay of the start of the transformation (reduction) and movement of the image data to be combined is propagated to subsequent timings of combination and consequently the combination is successively performed with the deviated timing of the screen formation.
This is because, as described above, in the conventional data combining apparatus combining image data and graphics data etc. to form the screen, the accurately timing adjustment for combining the graphics to be combined and the image data to be combined does not performed.
Particularly, in the case where each of the control units (the screen formation control unit, the image control unit, and the graphics control unit) is configured by a plurality of processors, in the case where communication paths between the control units or between a control unit and a processing unit (the transformation and movement processing unit or the graphics generation unit) are connected by buses including delay, or in the case where the communication takes time, the delay easily occurs until a notice of change of the screen formation is actually reflected to the transformation and movement processing unit or the graphics generation unit.
Also, in the case where each of the control units or processing units is configured by software, the delay easily occurs.
Further, if the image data and the graphics are asynchronous, the case where the timings of the two are deviated is similar to the case where any one of them is delayed.
Therefore, in the conventional data combining apparatus, it is difficult to realize a screen formation including transformation and movement (arrangement) of the data to be combined and specifically needing to adjust the timing such that the transformation and amount of movement and the arrangement position are changed continuously.
Namely, in the OSD, restrictions that the graphics and the image data to be combined have no relation in time are imposed.
In a screen formation combining a graphics at a region without the transformed and moved (arranged) image data, use is made under the restriction that the screen formation be one in which the amount of transformation and movement and the arrangement position do not change continuously.
Under the latter restriction, when the screen formation is switched, due to the deviation of timing of the image data and the graphics, a contradictory combined image is displayed.